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Abstract: The role of telerobotics in space exploration as placing human cognition on other worlds 
is limited almost entirely by the speed of light, and the consequent communications latency that 
results from large distances. This latency is the time delay between the human brain at one end, and 
the telerobotic effector and sensor at the other end. While telerobotics and virtual presence is a 
technology that is rapidly becoming more sophisticated, with strong commercial interest on the 
Earth, this time delay, along with the neurological timescale of a human being, quantitatively defines 
the cognitive horizon for any locale in space. That is, how distant can an operator be from a robot and 
not be significantly impacted by latency? We explore that cognitive timescale of the universe, and 
consider the implications for telerobotics, human space flight, and participation by larger numbers of 
people in space exploration. We conclude that, with advanced telepresence, sophisticated robots 
could be operated with high cognition throughout a lunar hemisphere by astronauts within a station 
at an Earth-Moon LI or L2 venue. Likewise, complex telerobotic servicing of satellites in 
geosynchronous orbit can be carried out from suitable terrestrial stations. 

"It's the latency, stupid" 

Stuart Chesire, “Wizard without Portfolio”, Apple Computer, Inc. 


1. Introduction 

The contrast between human space flight, on the one hand, and human space 
exploration on the other, is critical and poorly understood. The former is essentially 
about launching astronauts into space. The latter is about, as expressed by the MIT 
space policy group, "an expansion of the realm of human experience”, and may or 
may not require astronauts . 1 In the case of human space flight, it is largely just the 
astronauts whose realm of experience is being expanded. In the case of space 
exploration, it may be many more people, most of whom may never leave the Earth. 
This distinction is unsettling to many, as the concept of human exploration seems 
inextricably connected with "going there ". 2 Bringing human experience to other 
worlds without bringing humans is the promise of telerobotics, and what is called 
virtual presence. 


2. Telerobotics, Virtual Presence, and the Importance of Latency 

The rapid advance of telerobotics, both commercial and scientific, prompts careful 
examination of the contrast between human space flight and human space 
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exploration. Telerobotics is, in this sense, about an approach toward "virtual 
reality”, and the extent to which a human being can visit with and interact with 
another place without actually being there. That is no less than human space 
exploration without astronauts. There are a number of important aspects of human 
space flight that cannot be addressed in this way. For example, human physiology in 
the space environment, and the potential medical benefits that such work brings 
back to Earth, if not an ultimate task of expanding humanity. But for the larger 
question of understanding and experiencing other places, the importance of 
physically being there is rapidly becoming less than it used to be. Technologically, 
our capabilities in this area are in their infancy, though we have made great 
progress in high-bandwidth communication links, high-quality imaging sensors, and 
dexterous manipulators. These manipulators could be enhanced with haptic sensors 
that bring our sense of touch to other places as well. If we desire to duplicate human 
senses more completely, chemical sensors for smell and taste, and microphones for 
hearing could, in principle, also be added, though may well not be necessary to the 
job that needs to be done. All these sensors and manipulators can bring our 
awareness and responsiveness to other places. Progress on such telerobotics is 
relentless, and advances in at least built-in autonomy and self-prediction routines 
can be expected to be well-represented by Moore's Law of exponential growth in 
digital switch density. Having astronauts assisted by telerobots is a key part of 
current NASA space policy, most recently expressed in the Enabling Technology 
Development and Demonstration Program, as well as the draft technology 
roadmaps from the Office of the Chief Technologist . 3 The expanded use of 
telepresence to achieve key goals in support of human space exploration was 
endorsed by the Augustine Committee . 4 Telerobotic exploration was a near-term 
goal for the Flexible Path strategy they considered a preferable option for space 
exploration . 5 

This essay makes the case that a fundamental difference between telerobotics and 
astronauts on-site for space exploration is latency - the time delay between an event 
and its observation. This essay is not intended to be a human-versus-robot 
discussion, but rather what might be better posed as a humans-versus-spacesuits 
discussion; that is, how humans, at a site where they are not latency-limited, can 
most effectively experience and explore. Latency is, in many respects, the immutable 
savior of human space flight as rationale for human space exploration. Advances in 
telerobotics will include impressive autonomous abilities. These will be used for 
supervisory control with intelligent assistant software mitigating the latency in the 
interaction of human and robot, thereby extending our cognitive reach. Efforts on 
Robotnaut have been enabling in this regard . 6 But extending the senses and 
dexterity of a human being is a lot easier than duplicating in hardware the real-time 
decision-making power and common sense of a human brain. Latency is the 
property that truly constrains human telerobotic space exploration, or at least 
experience, of the cosmos. Indeed, when the latency is sufficiently low and 
telepresence sufficiently sophisticated, the experience of an astronaut on-site, 
encased within a constraining and sense-dulling spacesuit, is likely to be far less 
"real" than that for humans immersed in high-quality telepresence. 
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The importance of latency in planetary exploration has been addressed by a number 
of workers. Spudis and Taylor offer a thoughtful assessment of telerobotics, and the 
importance of low latency for lunar science. 7 While they conclude that low-latency 
telepresence is not essential for science, high latency may lead the operator to 
concentrate more on the mechanical aspects of the work than on the intellectual aspects 
of it. In this respect, minimizing latency in telepresence may have value that goes beyond 
simple reduction of time to accomplish a task. Quantitative assessments of the impact of 
latency on exploration efficiency have been done using Earth-based exploration analogs. 
Snook et al. evaluated the fidelity of science tasks done in three ways — (1) shirt-sleeved 
human, (2) human in EVA suit, and (3) a “2015-class” telerobot with many-second 
latency, and they compared the time required in each case. In this comparison, they 
found that the EVA-suited scientist could get things done about 5 times faster than 
this particular "telerobot", and the shirt-sleeved human could do it a lot faster. Of 
course, a shirt-sleeved human is irrelevant to most space exploration, but is an 
illuminating comparison. If they added more latency, corresponding to Earth-to- 
Mars, they figured a factor of more like 25 between an on-site astronaut and a 
telerobot. Furthermore, since a "2015-class" telerobot was, they figured, about ten 
times more capable than a "2002-class" telerobot, a space-suited human on Mars 
could perform work about 250 times more efficiently than the early 21 st -century 
telerobot on Mars controlled from the Earth. But if you reduce the latency, and 
possess a technologically more sophisticated robot, that factor will be reduced. 
Latency is by far the most important effect here in cognitive impediment. 

In planetary science, the importance of sample return is well understood. Humans 
on site can return with a large mass of samples, primarily because a return vehicle 
is a requirement for human exploration, and such a return vehicle must necessarily 
have a large mass and volume capability. If returning a large mass of samples is a 
major priority, there is no reason why a telerobotic explorer cannot have that 
capability. 

The implications of this issue for technology investment priorities of space agencies 
are profound. Space technology funding is under constant pressure with respect to, 
for example, on-going agency projects and programs. Thus, the challenge becomes 
whether increasingly limited funds will be invested to enable a tiny handful of 
humans (aka, astronauts) to explore exotic worlds directly or whether those funds 
will be used to enable an alternative vision for exploration by a larger number of 
people via advanced telepresence. Space exploration advocates readily dismiss 
generic "astronauts versus robots” controversies; far less easily dismissed in many 
cases is "astronauts versus low-latency telepresence." In an era of limited funding, 
the latter may well be a higher priority to enable human space exploration. 


3. Latency as Dictated by the Speed of Light in Space Exploration 
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For space exploration, latency, otherwise more broadly known in the 
communications industry as "lag”, "ping”, or simply time delay, in this case limited 
by the speed of light, is what keeps virtual reality in space unrealized at an 
important level. For the Moon, the two-way latency is about 2.6 seconds, even 
assuming that other elements of communication latency, such as serialization, 
packet switching, or queueing are kept to zero. You push a button in Pasadena and, 
at best, you see the shovel on the lunar bulldozer drop 2.6 seconds later. On Mars, 
make that 3 minutes or as many as 22 minutes! The quote in the introduction to this 
paper by Stuart Chesire just expresses the fact that latency, rather than bandwidth, 
can be the limiting feature of a communication channel when it comes to getting 
things done. 

What telerobotics does is to put human consciousness and capability at places that 
may not be convenient, or even safe, for human beings. It can be not only about 
safety for human beings but, for the more difficult destinations, about making 
operations affordable there. What telerobotics does not do in space exploration is to 
feature personal courage, risk, and human endurance in obvious ways. Those 
qualities of human brio are valued elements of a strong society. Although those 
qualities can be achieved in many other ways, telepresence in space may never 
entirely replace human space flight as an important international endeavor. For 
example, telerobotics bears on space colonization and settlement, but only in the 
way it might help people move somewhere else. Such settlement and colonization 
might be of value, though has never been identified as a formal priority by any 
nation. 

Some may contend that with regard to capability, an in situ human cannot be 
replaced by a telerobot. That is, virtual presence can never displace feet on the 
ground or fingers in the regolith. Judgment, creativity, dexterity, and precision can 
only come from human flesh onsite, and will never be transmitted over 
communication channels. But this is a lesson derived from historical exploration and 
one that may not pertain anymore. It is getting less true rapidly, a fact that is 
especially evident to a younger generation that has grown up with advanced 
communication and virtual-presence technologies. In any case, bulky space suits, 
helmets, gloves, and boots seriously inhibit any real experience of the environment 
and actually cause sensory deprivation. 

Again, to the extent that exploration is about discovery and doing things, being there 
virtually, even with high latency, but with sensors that are far better than those on 
our own bodies, and manipulators that can operate more precisely and deftly than 
our fingers, will eventually eclipse astronauts being there physically. The evolution 
of exploration, driven by our newfound technological skills, is happening with 
astonishing rapidity. It is something that many have not yet recognized and may 
have a hard time accepting. 

In many respects the role of telerobotics in the evolution of space exploration is 
reminiscent of the fear of automation fifty years ago, when inhuman machines made 
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of wire and gears were going to replace dedicated craftsmen whose forefathers had 
passed on their deep wisdom to them. Their skilled hands and keen eyes would, no 
doubt, end up out on the sidewalk selling pencils. At that time, the very idea that 
manufacturing by pairs of deft, but calloused, hands would be largely taken over by 
machines was both ludicrous and frightening. It is no longer. We understand that 
those skills are still of value in creating and controlling these machines. Our cultural 
perspective on manufacturing evolved dramatically, as our cultural perspective on 
space exploration now has to. As manufacturing by humans came to rely on human- 
controlled robots, so human exploration now appears to be evolving . . . and rapidly. 

The way that latency affects teleoperation and telepresence is a well-studied matter 
of cognitive neuroscience. See for example the reviews by Straube 9 and Bunge. 10 
Mental chronometry, as it is termed, is about how response time in perceptual- 
motor tasks affects understanding of the content, time duration, and sequencing of 
cognitive operations. It is clear that cognitive operations are seriously impacted as 
latency increases above a certain length of time. That limiting time scale, multiplied 
by the speed of light, can be termed the "cognitive scale of the universe." It is the 
distance over which our real-time cognition - that is, real-time human thinking and 
mental response - can be effectively transmitted. 

There is no question that humans have the ability to control telerobotics in 
spacecraft at large distances. The two Voyager spacecraft, now roughly a hundred 
astronomical units away, presenting a two-way latency of more than a day, are still 
being directed by dedicated (and patient!] human explorers back on Earth, and 
continuing their long list of accomplishments. These craft are considered our 
farthest reach in space exploration and have been labeled as a "grand gesture of the 
Third Age of Discovery,’’ 11 But that latency delay is hardly matched to the cognitive 
abilities of these human explorers. They push the button, and wait, and wait ... 

This essay in no way adequately reviews the enormous amount of work in this field, 
which has examined the effects of latency on different functions. The impact of 
latency on perceptual-motor tasks depends strongly on exactly what one is trying to 
accomplish. Operation of the Voyager spacecraft is pretty simple, and hardly tasks 
human cognition, even at shorter distances. Other efforts could be far more 
challenging. 


4. Cognitive Time Scales in Space Exploration 

The cognitive time scale of the universe can be defined as the distance at which 
perceptual-motor tasks are equally limited by human neurophysiology, synaptic 
linkages and the speed of light. We can note a few relevant time scales. 

• 20-40 ms - two-way neural signal transmission between brain and fingertip 
® 150 ms — recommended two-way maximum time delay for telephone service 
« 200 ms — human eye-hand reaction time 
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• 300 to 400 ms — the blink of a human eye 

• 400 ms — time for the fastest American baseball pitches to reach the strike zone 

In particular, for online gaming, for which low latency is particularly critical, the 
two-way latency numbers are 

• 50-60 ms - limit of delay detection 

• 200 ms - delays becomes noticeable 

• 300-500 ms - games become unpleasant, even unplayable 

To these we can add experience from commercial activities that are increasingly 
reliant on telepresence. Robotic telepresence in mining equipment typically involves 
two-way latencies of no more than a few hundred milliseconds. For transcontinental 
surgery by telepresence, now widely accepted by the medical community, many 
surgeons can work successfully with a latency of 300 ms. Properly trained surgeons 
can even do precision surgical tasks with 500 ms of latency. 12 Piloting of military 
drone attack aircraft can involve two-way latencies of 1000-2000 ms, although 
those unimpressive latencies are dictated by being able to land, rather than to fly. 
The latter seems to be more forgiving of latency. The relevance of latencies to space 
exploration telerobotics have been investigated by a number of workers. 13 ' 14 - 15 ' 16 - 17 

Full haptic teleoperation, in which you do things by "feel" as well as by visual cues, 
requires ~100 ms latency or less. That is, getting information by feel on 
directionality and "slippage" needs fairly high performance. Low latency thus can 
provide a richly endowed situational awareness. Haptic (vibrotactile/force- 
feedback) sensors are considered to be generally useless for latencies of order one 
second or longer. 

Clearly, the cognitive scale of the universe is going to be in this range of light travel 
time of order a few hundreds of milliseconds. Again, we assume that we are limited 
entirely by light time, in that fast packet switching and dedicated communication 
systems can apply to space efforts. Of course for many modern space 
communication systems such as NASA's Tracking and Data Relay Satellite System 
(TDRSS), where the conduit is shared, and bitrates for large files is more important 
than for small files, latency is less important than bandwidth. 

Two-way light-time latencies between relevant points in our solar system are: 

• 2600 ms Earth-to-Moon 

• 410 ms Earth-Moon LI or L2-to-lunar surface 

• 240 ms Earth-to-GEO 

» 130 ms one side of the Earth to the other (circumferentially) 

• 130 ms Deimos-to-Mars (Deimos is very roughly aereostationary) 

It is clear from this table and the cognitive time scales above that the distance from 
the Earth to the Moon is considerably larger than what we can call the cognitive 
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scale of the universe, which is a very few hundred light milliseconds across. As a 
result, latency renders telerobots on the Moon controlled from the Earth somewhat 
less effective when short response times are important. That does not mean that we 
cannot teleoperate equipment in deep space from the Earth [as we do with 
Voyager), but just that humans are not completely cognitively "on site” when they 
do. At the other extreme, for example, we point out that with sophisticated 
telepresence and with regard to equipment operation, there is little obvious value 
for humans to be closer to a target site than light can travel in ~50 ms (~15,000 
km): human perception and response is typically not much faster than this. 

We are not suggesting "total immersion" telepresence that would allow for a “virtual 
reality” experience. Not only are the technologies not yet available, but they are not 
necessary for enabling operation. Total sensory inclusion (smells of abraded rock, 
and the feel of dust falling on ones head, for example) is not particularly important 
for terrestrial mining, and are unlikely to be for most tasks in space. 


5. Earth-Moon Lagrange Points for Cognitive Control on the Lunar Surface 

Of special importance here is the fact that Earth-Moon LI or L2 libration points are 
just inside the cognitive horizon of the lunar surface. Earth-Moon LI, for example, is 
61,500 km Earthward from the Moon, about 16% of the Earth-Moon distance from 
the Moon. As a result, teleoperation of nearside lunar surface equipment, such as 
sample collection and inspection, mining and refining for ISRU, and lunar base 
development, can be carried out capably via telerobots with nearly full cognition 
from astronauts orbiting LI. Such an Earth-Moon LI orbit has been the subject of 
many concept studies for human space habitation. See, for example, the NASA 
Decadal Planning Team studies from 1999. 18 , 19 From these studies, Earth-Moon LI 
is understood to be a credible and enabling near-term destination for human space 
travel. 20 

Not only is Earth-Moon LI an enabling place for development, servicing, and 
depoting of equipment for cis-lunar space, but also is a site that contemporary 
EELV-Heavy launchers and habitat technologies should be able to support. Lunar 
nearside sites will generally have continuous coverage by telerobotic control from 
such an orbit, although the line-of-sight availability to the lunar poles will depend on 
the particular orbit. Although Earth-Moon Lagrange point operations figured 
prominently in the industry Concept Exploration and Refinement (CE&R) studies 
that predated the 2005 Exploration Systems and Architecture Study (ESAS), CE&R 
studies were largely ignored in what turned out to be the heavily lunar surface- 
centric Constellation architecture. 21 

We note that Earth-Moon L2 is similarly enabling for the lunar farside. That side 
does not allow direct telerobotic control from the Earth without a communication 
relay. While the South Pole Aitken basin on the far side is a key priority science 
target, the far side has many fewer such targets than the nearside (e.g. ESAS 4.3. 6. 3), 
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